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Abstract 

Software-defined  networking  (SDN)  is  revolutionizing  the  net¬ 
working  industry,  but  current  SDN  programming  platforms  do  not 
provide  automated  mechanisms  for  updating  global  configurations 
on  the  fly.  Implementing  updates  by  hand  is  challenging  for  SDN 
programmers  because  networks  are  distributed  systems  with  hun¬ 
dreds  or  thousands  of  interacting  nodes.  Even  if  initial  and  final 
configurations  are  correct,  naively  updating  individual  nodes  can 
lead  to  incorrect  transient  behaviors,  including  loops,  black  holes, 
and  access  control  violations.  This  paper  presents  an  approach  for 
automatically  synthesizing  updates  that  are  guaranteed  to  preserve 
specified  properties.  We  formalize  network  updates  as  a  distributed 
programming  problem  and  develop  a  synthesis  algorithm  based  on 
counterexample-guided  search  and  incremental  model  checking. 
We  describe  a  prototype  implementation,  and  present  results  from 
experiments  on  real-world  topologies  and  properties  demonstrating 
that  our  tool  scales  to  updates  involving  over  one-thousand  nodes. 
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1.  Introduction 

Software-defined  networking  (SDN)  is  a  new  paradigm  in  which 
a  logically-centralized  controller  manages  a  collection  of  pro¬ 
grammable  switches.  The  controller  responds  to  events  such  as 
topology  changes,  shifts  in  traffic  load,  or  new  connections  from 
hosts,  by  pushing  forwarding  rules  to  the  switches,  which  process 
packets  efficiently  using  specialized  hardware.  Because  the  con¬ 
troller  has  global  visibility  and  full  control  over  the  entire  network, 
SDN  makes  it  possible  to  implement  a  wide  variety  of  network 
applications  ranging  from  basic  routing  to  traffic  engineering,  data¬ 
center  virtualization,  fine-grained  access  control,  etc.  [6].  SDN  has 
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been  used  in  production  enterprise,  datacenter,  and  wide-area  net¬ 
works,  and  new  deployments  are  rapidly  emerging. 

Much  of  SDN’s  power  stems  from  the  controller's  ability  to 
change  the  global  state  of  the  network.  Controllers  can  set  up  end- 
to-end  forwarding  paths,  provision  bandwidth  to  optimize  utiliza¬ 
tion,  or  distribute  access  control  rules  to  defend  against  attacks. 
However,  implementing  these  global  changes  in  a  running  network 
is  not  easy.  Networks  are  complex  systems  with  many  distributed 
switches,  but  the  controller  can  only  modify  the  configuration  of 
one  switch  at  a  time.  Hence,  to  implement  a  global  change,  an 
SDN  programmer  must  explicitly  transition  the  network  through 
a  sequence  of  intermediate  configurations  to  reach  the  intended  fi¬ 
nal  configuration.  The  code  needed  to  implement  this  transition  is 
tedious  to  write  and  prone  to  error — in  general,  the  intermediate 
configurations  may  exhibit  new  behaviors  that  would  not  arise  in 
the  initial  and  final  configurations. 

Problems  related  to  network  updates  are  not  unique  to  SDN. 
Traditional  distributed  routing  protocols  also  suffer  from  anomalies 
during  periods  of  reconvergence,  including  transient  forwarding 
loops,  blackholes,  and  access  control  violations.  For  users,  these 
anomalies  manifest  themselves  as  service  outages,  degraded  per¬ 
formance,  and  broken  connections.  The  research  community  has 
developed  techniques  for  preserving  certain  invariants  during  up¬ 
dates  [9,  32,  36],  but  none  of  them  fully  solves  the  problem,  as  they 
are  limited  to  specific  protocols  and  properties.  For  example,  con¬ 
sensus  routing  uses  distributed  snapshots  to  ensure  connectivity,  but 
only  applies  to  the  Border  Gateway  Protocol  (BGP)  [16], 

It  might  seem  that  SDN  would  exacerbate  update-related  prob¬ 
lems  by  making  networks  even  more  dynamic — in  particular,  most 
current  platforms  lack  mechanisms  for  implementing  updates  in  a 
graceful  way.  However,  SDN  offers  opportunities  to  develop  high- 
level  abstractions  for  implementing  updates  automatically  while 
preserving  key  invariants.  The  authors  of  B4 — the  controller  man¬ 
aging  Google’s  world-wide  inter-datacenter  network — describe  a 
vision  where:  “multiple,  sequenced  manual  operations  [are]  not  in¬ 
volved  [in]  virtually  any  management  operation”  [14]. 

Previous  work  proposed  the  notion  of  a  consistent  update  [33], 
which  ensures  that  every  packet  is  processed  either  using  the  initial 
configuration  or  the  final  configuration  but  not  a  mixture  of  the  two. 
Consistency  is  a  powerful  guarantee  preserving  all  safety  proper¬ 
ties,  but  it  is  expensive.  The  only  general  consistent  update  mech¬ 
anism  is  two-phase  update,  which  tags  packets  with  versions  and 
maintains  rules  for  the  initial/final  configurations  simultaneously. 
This  leads  to  problems  on  switches  with  limited  memory  and  can 
also  make  update  time  slower  due  to  the  high  degree  of  rule  churn. 

We  propose  an  alternative.  Instead  of  forcing  SDN  operators 
to  implement  updates  by  hand  (as  is  typically  done  today),  or  us¬ 
ing  powerful  but  expensive  mechanisms  like  two-phase  update,  we 
develop  an  approach  for  synthesizing  correct  update  programs  ef¬ 
ficiently  and  automatically  from  formal  specifications.  Given  ini¬ 
tial  and  final  configurations  and  a  Linear  Temporal  Logic  (LTL) 
property  capturing  desired  invariants  during  the  update,  we  either 
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Figure  1 :  Example  topology. 
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Figure  2:  Example  naive  (blue/solid-line),  two-phase  (green/solid-bar),  and  or¬ 
dering  (red/dashed)  updates:  (a)  probes  received;  (b)  per-switch  rule  overhead. 


generate  an  SDN  program  that  implements  the  initial-to-final  tran¬ 
sition  while  ensuring  that  the  property  is  never  violated,  or  fail  if  no 
such  program  exists.  Importantly,  because  the  synthesized  program 
is  only  required  to  preserve  the  specified  properties,  it  can  leverage 
strategies  that  would  be  ruled  out  in  other  approaches.  For  example, 
if  the  programmer  specifies  a  trivial  property,  the  system  can  update 
switches  in  any  order.  However,  if  she  specifies  a  more  complex 
property  (e.g.  firewall  traversal)  then  the  space  of  possible  updates 
is  more  constrained.  In  practice,  our  synthesized  programs  require 
less  memory  and  communication  than  competing  approaches. 

Programming  updates  correctly  is  challenging  due  to  the  con¬ 
currency  inherent  in  networks — switches  may  interleave  packet  and 
control  message  processing  arbitrarily.  Hence,  programmers  must 
carefully  consider  all  possible  event  orderings,  inserting  synchro¬ 
nization  primitives  as  needed.  Our  algorithm  works  by  searching 
through  the  space  of  possible  sequences  of  individual  switch  up¬ 
dates,  learning  from  counterexamples  and  employing  an  incremen¬ 
tal  model  checker  to  re-use  previously  computed  results.  Our  model 
checker  is  incremental  in  the  sense  that  it  exploits  the  loop-freedom 
of  correct  network  configurations  to  enable  efficient  re-checking 
of  properties  when  the  model  changes.  Because  the  synthesis  al¬ 
gorithm  poses  a  series  of  closely-related  model  checking  ques¬ 
tions,  the  incrementality  yields  enormous  performance  gains  on 
real-world  update  scenarios. 

We  have  implemented  the  algorithm  and  heuristics  to  further 
speed  up  synthesis  and  eliminate  spurious  synchronization.  We 
have  interfaced  the  tool  with  Frenetic  [8],  synthesized  updates  for 
OpenFlow  switches,  and  used  our  system  to  process  actual  traffic 
generated  by  end-hosts.  We  ran  experiments  on  a  suite  of  real-world 
topologies,  configurations,  and  properties — our  results  demonstrate 
the  effectiveness  of  synthesis,  which  scales  to  over  one-thousand 
switches,  and  incremental  model  checking,  which  outperforms  a 
popular  symbolic  model  checker  used  in  batch  mode,  and  a  state- 
of-the-art  network  model  checker  used  in  incremental  mode. 

In  summary,  the  main  contributions  of  this  paper  are: 

•  We  investigate  using  synthesis  to  automatically  generate  net¬ 
work  updates  (§2). 

•  We  develop  a  simple  operational  model  of  SDN  and  formalize 
the  network  update  problem  precisely  (§3). 

•  We  design  a  counterexample-guided  search  algorithm  that 
solves  instances  of  the  network  update  problem,  and  prove  this 
algorithm  to  be  correct  (§4). 

•  We  present  an  incremental  LTL  model  checker  for  loop-free 
models  (§5). 

•  We  describe  an  OCaml  implementation  with  backends  to  third- 
party  model  checkers  and  conduct  experiments  on  real-world 
networks  and  properties,  demonstrating  strong  performance  im¬ 
provements  (§6).  f 

Overall,  our  work  takes  a  challenging  network  programming  prob¬ 
lem  and  automates  it,  yielding  a  powerful  tool  for  building  dynamic 
SDN  applications  that  ensures  correct,  predictable,  and  efficient 
network  behavior  during  updates. 


2.  Overview 

To  illustrate  key  challenges  related  to  network  updates,  consider 
the  network  in  Figure  1 .  It  represents  a  simplified  datacenter  topol¬ 
ogy  [1]  with  core  switches  (Cl  and  C2),  aggregation  switches  (A1 
to  A4),  top-of-rack  switches  (T1  to  T4),  and  hosts  (HI  to  H4).  Ini¬ 
tially,  we  configure  switches  to  forward  traffic  from  HI  to  H3  along 
the  solid/red  path:  T1-A1-C1-A3-T3.  Later,  we  wish  to  shift  traffic 
from  the  red  path  to  the  dashed/green  path,  T1-A1-C2-A3-T3  (per¬ 
haps  to  take  Cl  down  for  maintenance).  To  implement  this  update, 
the  operator  must  modify  forwarding  rules  on  switches  A1  and  C2, 
but  note  that  certain  update  sequences  break  connectivity — e.g.,  up¬ 
dating  A1  followed  by  C2  causes  packets  to  be  forwarded  to  C2  be¬ 
fore  it  is  ready  to  handle  them.  Figure  2(a)  demonstrates  this  with  a 
simple  experiment  performed  using  our  system.  Using  the  Mininet 
network  simulator  and  OpenFlow  switches,  we  continuously  sent 
ICMP  (ping)  probes  during  a  “naive”  update  (blue/solid  line)  and 
the  ordering  update  synthesized  by  our  tool  (red/dashed  line).  With 
the  naive  update,  100%  of  the  probes  are  lost  during  an  interval, 
while  the  ordering  update  maintains  connectivity. 

Consistency.  Previous  work  [33]  introduced  the  notion  of  a  con¬ 
sistent  update  and  also  developed  general  mechanisms  for  ensuring 
consistency.  An  update  is  said  to  be  consistent  if  every  packet  is 
processed  entirely  using  the  initial  configuration  or  entirely  using 
the  final  configuration,  but  never  a  mixture  of  the  two.  For  exam¬ 
ple,  updating  A1  followed  by  C2  is  not  consistent  because  packets 
from  HI  to  H3  might  be  dropped  instead  of  following  the  red  path 
or  the  green  path.  One  might  wonder  whether  preserving  consis¬ 
tency  during  updates  is  important,  as  long  as  the  network  eventu¬ 
ally  reaches  the  intended  configuration,  since  most  networks  only 
provide  best-effort  packet  delivery.  While  it  is  true  that  errors  can 
be  masked  by  protocols  such  as  TCP  when  packets  are  lost,  there 
is  growing  interest  in  strong  guarantees  about  network  behavior. 
For  example,  consider  a  business  using  a  firewall  to  protect  internal 
servers,  and  suppose  that  they  decide  to  migrate  their  infrastructure 
to  a  virtualized  environment  like  Amazon  EC2.  To  ensure  that  this 
new  deployment  is  secure,  the  business  would  want  to  maintain  the 
same  isolation  properties  enforced  in  their  home  office.  However,  a 
best-effort  migration  strategy  that  only  eventually  reaches  the  tar¬ 
get  configuration  could  step  through  arbitrary  intermediate  states, 
some  of  which  may  violate  this  property. 

Two-Phase  Updates.  Previous  work  introduced  a  general 
consistency-preserving  technique  called  two-phase  update  [33]. 
The  idea  is  to  explicitly  tag  packets  upon  ingress  and  use  these 
version  tags  to  determine  which  forwarding  rules  to  use  at  each 
hop.  Unfortunately,  this  has  a  significant  cost.  During  the  transition, 
switches  must  maintain  forwarding  rules  for  both  configurations, 
effectively  doubling  the  memory  requirements  needed  to  complete 
the  update.  This  is  not  always  practical  in  networks  where  the 
switches  store  forwarding  rules  using  ternary  content-addressable 
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memories  (TCAM),  which  are  expensive  and  power-hungry.  Fig¬ 
ure  2(b)  shows  the  results  of  another  simple  experiment  where  we 
measured  the  total  number  of  rules  on  each  switch:  with  two-phase 
updates,  several  switches  have  twice  the  number  of  rules  compared 
to  the  synthesized  ordering  update.  Even  worse,  it  takes  a  non¬ 
trivial  amount  of  time  to  modify  forwarding  rules — sometimes  on 
the  order  of  10ms  per  rule  [15] !  Hence,  because  two-phase  updates 
modify  a  large  number  of  rules,  they  can  increase  update  latency. 
These  overheads  can  make  two-phase  updates  a  non-starter. 

Ordering  Updates.  Our  approach  is  based  on  the  observation  that 
consistent  (two-phase)  updates  are  overkill  in  many  settings.  Some¬ 
times  consistency  can  be  achieved  by  simply  choosing  a  correct 
order  of  switch  updates.  We  call  this  type  of  update  an  ordering 
update.  For  example,  to  update  from  the  red  path  to  the  green  path, 
we  can  update  C2  followed  by  Al.  Moreover,  even  when  we  can¬ 
not  achieve  full  consistency,  we  can  often  still  obtain  sufficiently 
strong  guarantees  for  a  specific  application  by  carefully  updating 
the  switches  in  a  particular  order.  To  illustrate,  suppose  that  instead 
of  shifting  traffic  to  the  green  path,  we  wish  to  use  the  blue  (dashed- 
and-dotted)  path:  T1-A2-C1-A4-T3.  It  is  impossible  to  transition 
from  the  red  path  to  the  blue  path  by  ordering  switch  updates  with¬ 
out  breaking  consistency:  we  can  update  A2  and  A4  first,  as  they 
are  unreachable  in  the  initial  configuration,  but  if  we  update  T1  fol¬ 
lowed  by  Cl,  then  packets  can  traverse  the  path  T1-A2-C1-A3-T3, 
while  if  we  update  Cl  followed  by  Tl,  then  packets  can  traverse  the 
path  T1-A1-C1-A4-T3.  Neither  of  these  alternatives  is  allowed  in  a 
consistent  update.  This  failure  to  find  a  consistent  update  hints  at  a 
solution:  if  we  only  care  about  preserving  connectivity  between  HI 
and  H3,  then  either  path  is  actually  acceptable.  Thus,  either  updat¬ 
ing  Cl  before  Tl,  or  Tl  before  Cl  would  work.  Hence,  if  we  relax 
strict  consistency  and  instead  provide  programmers  with  a  way  to 
specify  properties  that  must  be  preserved  across  an  update,  then  or¬ 
dering  updates  will  exist  in  many  situations.  Recent  work  [15,  25] 
has  explored  ordering  updates,  but  only  for  specific  properties  like 
loop-freedom,  blackhole-freedom,  drop-freedom,  etc.  Rather  than 
handling  a  fixed  set  of  “canned”  properties,  we  use  a  specification 
language  that  is  expressive  enough  to  encode  these  properties  and 
others,  as  well  as  conjunctions/disjunctions  of  properties — e.g.  en¬ 
forcing  loop-freedom  and  service-chaining  during  an  update. 

In-flight  Packets  and  Waits.  Sometimes  an  additional  synchro¬ 
nization  primitive  is  needed  to  generate  correct  ordering  updates 
(or  correct  two-phase  updates,  for  that  matter).  Suppose  we  want 
to  again  transition  from  the  red  path  to  blue  one,  but  in  addition  to 
preserving  connectivity,  we  want  every  packet  to  traverse  either  A2 
or  A3  (this  scenario  might  arise  if  those  switches  are  actually  mid- 
dleboxes  which  scrub  malicious  packets  before  forwarding).  Now 
consider  an  update  that  modifies  the  configurations  on  A2,  A4,  Tl, 
Cl,  in  that  order.  Between  the  time  that  we  update  Tl  and  Cl,  there 
might  be  some  packets  that  are  forwarded  by  Tl  before  it  is  up¬ 
dated,  and  are  forwarded  by  Cl  after  it  is  updated.  These  packets 
would  not  traverse  A2  or  A3,  and  so  indicate  a  violation  of  the  spec¬ 
ification.  To  fix  this,  we  can  simply  pause  after  updating  T 1  until 
any  packets  it  previously  forwarded  have  left  the  network.  We  thus 
need  a  command  “wait”  that  pauses  the  controller  for  a  sufficient 
period  of  time  to  ensure  that  in-flight  packets  have  exited  the  net¬ 
work.  Hence,  the  correct  update  sequence  for  this  example  would 
be  as  above,  with  a  “wait”  between  Tl  and  Cl.  Note  that  two- 
phase  updates  also  need  to  wait,  once  per  update,  since  we  must 
ensure  that  all  in-flight  packets  have  left  the  network  before  delet¬ 
ing  the  old  version  of  the  rules  on  switches.  Other  approaches  have 
traded  off  control-plane  waiting  for  stronger  consistency,  e.g.  [24] 
performs  updates  in  “rounds”  that  are  analogous  to  “wait”  com¬ 
mands,  and  Consensus  Routing  [16]  relies  on  timers  to  obtain  wait¬ 
like  functionality.  Note  that  the  single-switch  update  time  can  be 


on  the  order  of  seconds  [15,  22],  whereas  typical  datacenter  transit 
time  (the  time  for  a  packet  to  traverse  the  network)  is  much  lower, 
even  on  the  order  of  microseconds  [3],  Hence,  waiting  for  in-flight 
packets  has  a  negligible  overall  effect.  In  addition,  our  reachability- 
based  heuristic  eliminates  most  waits  in  practice. 

Summary.  This  paper  presents  a  sound  and  complete  algorithm 
and  implementation  for  synthesizing  a  large  class  of  ordering  up¬ 
dates  efficiently  and  automatically.  The  updates  we  generate  ini¬ 
tially  modify  each  switch  at  most  once  and  “wait”  between  updates 
to  switches,  but  a  heuristic  removes  an  overwhelming  majority  of 
unnecessary  waits  in  practice.  For  example,  in  switching  from  the 
red  path  to  the  blue  path  (while  preserving  connectivity  front  HI  to 
H3,  and  making  sure  that  each  packet  visits  either  A3  or  A4),  our 
tool  produces  the  following  sequence:  update  A2,  then  A4,  then 
Tl,  then  wait,  then  update  Cl.  The  resulting  update  can  be  exe¬ 
cuted  using  the  Frenetic  SDN  platform  and  used  with  OpenFlow 
switches — e.g..  we  generated  Figure  2  (a-b)  using  our  tool. 

3.  Preliminaries  and  Network  Model 

To  facilitate  precise  reasoning  about  networks  during  updates,  we 
develop  a  formal  model  in  the  style  of  Chemical  Abstract  Ma¬ 
chine  [4],  This  model  captures  key  network  features  using  a  simple 
operational  semantics.  It  is  similar  to  the  one  used  by  [11],  but  is 
streamlined  to  model  features  most  relevant  to  updates. 

3.1  Network  Model 

Basic  structures.  Each  switch  sw,  port  pt,  or  host  h  is  identified 
by  a  natural  number.  A  packet  pkt  is  a  record  of  fields  containing 
header  values  such  as  source  and  destination  address,  protocol  type, 
and  so  on.  We  write  {/i;  •  •  • ;  /fc}  for  the  type  of  packets  having 
fields  fi  and  use  “dot”  notation  to  project  fields  from  records.  The 
notation  {r  with  /  =  v}  denotes  functional  update  of  r.f. 

Forwarding  Tables.  A  switch  configuration  is  defined  in  terms 
of  forwarding  rules,  where  each  rule  has  a  pattern  pat  specified 
as  a  record  of  optional  packet  header  fields  and  a  port,  a  list  of 
actions  act  that  either  forward  a  packet  out  a  given  port  ( fwd  pt) 
or  modify  a  header  field  ( f:=n ),  and  a  priority  that  disambiguates 
rules  with  overlapping  patterns.  We  write  {pi?;  /i  ;  fkfl}  for 
the  type  of  patterns,  where  the  question  mark  denotes  an  option 
type.  A  set  of  such  rules  ruls  forms  a  forwarding  table  tbl.  The 
semantic  function  [tM]  maps  packet-port  pairs  to  multisets  of  such 
pairs,  finding  the  highest-priority  rule  whose  pattern  matches  the 
packet  and  applying  the  corresponding  actions.  If  there  are  multiple 
matching  rules  with  the  same  priority,  the  function  is  free  to  pick 
any  of  them,  and  if  there  are  no  matching  rules,  it  drops  the  packet. 
The  forwarding  tables  collectively  define  the  network’s  data  plane. 

Commands.  The  control  plane  modifies  the  data  plane  by  is¬ 
suing  commands  that  update  forwarding  tables.  The  command 
(sw,  tbl)  replaces  the  forwarding  table  on  switch  sw  with  tbl  (we 
call  this  a  switch-granularity  update).  We  model  this  command  as 
an  atomic  operation  (it  can  be  implemented  with  OpenFlow  bun¬ 
dles  [31]).  Sometimes  switch  granularity  is  too  coarse  to  find  an  up¬ 
date  sequence,  in  which  case  one  can  update  individual  rules  ( rule- 
granularity ).  Our  tool  supports  this  finer-grained  mode  of  opera¬ 
tion,  but  since  it  is  not  conceptually  different  from  switch  granular¬ 
ity,  we  frame  most  of  our  discussion  in  terms  of  switch-granularity. 

To  synchronize  updates  involving  multiple  switches,  we  include 
a  wait  command.  In  the  model,  the  controller  maintains  a  natural- 
number  counter  known  as  the  current  epoch  ep.  Each  packet  is 
annotated  with  the  epoch  on  ingress.  The  control  command  incr 
increments  the  epoch  so  that  subsequent  incoming  packets  are  an¬ 
notated  with  the  next  epoch,  and  flush  blocks  the  controller  until  all 
packets  annotated  with  the  previous  epoch  have  exited  the  network. 
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Figure  3:  Network  model. 


We  introduce  a  command  wait  defined  as  incr-,  flush.  The  epochs 
are  included  in  our  model  solely  to  enable  reasoning.  They  do  not 
need  to  be  implemented  in  a  real  network — all  that  is  needed  is  a 
mechanism  for  blocking  the  controller  to  allow  a  flush  of  all  packets 
currently  in  the  network.  For  example,  given  a  topology,  one  could 
compute  a  conservative  delay  based  on  the  maximum  hop  count, 
and  then  implement  wait  by  sleeping,  rather  than  synchronizing 
with  each  switch.  Note  that  we  implicitly  assume  failure-freedom 
and  packet-forwarding  fairness  of  switches  and  links,  i.e.  there  is 
an  upper  bound  on  each  element’s  packet-processing  time. 

Elements.  The  elements  E  of  the  network  model  include 
switches  Si,  links  Lj,  and  a  single  controller  element  C,  and  a 
network  N  is  a  tuple  containing  these.  Each  switch  Si  is  encoded 
as  a  record  comprising  a  unique  identifier  sw,  a  table  tbl  of  pri¬ 
oritized  forwarding  rules,  and  a  multiset  prs  of  pairs  ( pkt,pt )  of 
buffered  packets  and  the  ports  they  should  be  forwarded  to  respec¬ 
tively.  Each  link  Lj  is  represented  by  a  record  consisting  of  two 
locations  loc  and  loc'  and  a  list  of  queued  packets  pkts,  where  a 
location  is  either  a  host  or  a  switch-port  pair.  Finally,  controller  C 
is  represented  by  a  record  containing  a  list  of  commands  cmds  and 
an  epoch  ep.  We  assume  that  commands  are  totally-ordered.  The 
controller  can  ensure  this  by  using  OpenFlow  barrier  messages. 

Operational  semantics.  Network  behavior  is  defined  by  small- 
step  operational  rules  in  Figure  3.  These  define  interactions  be¬ 
tween  subsets  of  elements,  based  on  OpenFlow  semantics  [28]. 
States  of  the  model  are  given  by  multisets  of  elements.  We  write 
{a:}  to  denote  a  singleton  multiset,  and  mi  tt)  m2  for  the  union  of 
multisets  mi  and  m2.  We  write  [ x ]  for  a  singleton  list,  and  l,@l2 
for  concatenation  of  l\  and  l2.  Each  transition  N  A-  N'  is  anno¬ 
tated,  with  o  being  either  an  empty  annotation,  or  an  observation 
(sw,  pt,  pkt )  indicating  the  location  and  packet  being  processed. 

The  first  rules  describe  date-plane  behavior.  The  In  rule  admits 
arbitrary  packets  into  the  network  from  a  host,  stamping  them 
with  the  current  controller  epoch.  The  OUT  rule  removes  a  packet 
buffered  on  a  link  adjacent  to  a  host.  PROCESS  processes  a  single 
packet  on  a  switch,  finding  the  highest  priority  rule  with  matching 
pattern,  applying  the  actions  of  that  rule  to  generate  a  multiset  of 
packets,  and  adding  those  packets  to  the  output  buffer.  FORWARD 


moves  a  packet  from  a  switch  to  the  adjacent  link.  The  final  rules 
describe  control-plane  behavior.  UPDATE  replaces  the  table  on  a 
single  switch.  INCR  increments  the  epoch  on  the  controller,  and 
FLUSH  blocks  the  controller  until  all  packets  in  the  network  are 
annotated  with  at  least  the  current  epoch  (ep(Es)  denotes  the 
smallest  annotation  on  any  packet  in  Es).  Finally,  CONGRUENCE, 
allows  any  sub-collection  of  network  elements  to  interact. 

3.2  Network  Update  Problem 

In  order  to  define  the  network  update  problem,  we  need  to  first 
define  traces  of  packets  flowing  through  the  network. 

Packet  traces.  Given  a  network  N,  our  operational  rules  can  gen¬ 
erate  sequences  of  observations.  However,  the  network  can  process 
many  packets  concurrently,  and  we  want  observations  generated  by 
a  single  packet.  We  define  a  successor  relation  C  for  observations 

ep 

(Definition  7,  Appendix  A).  Intuitively  o  C  o  if  the  network  can 
directly  produce  the  packet  in  o'  by  processing  o  in  the  epoch  ep. 

Definition  1  (Single-Packet  Trace).  Let  N  be  a  network.  A  se¬ 
quence  (oi  •  •  •  oi)  is  a  single-packet  trace  of  N  if  N  -4-  . . .  — ^ 
Nk  such  that  (oi  ■■■  oi)  is  a  subsequence  of(o'\  ■  ■  ■  o'k)  for  which 

•  every  observation  is  a  successor  of  the  preceding  observation 
in  monotonically  increasing  epochs,  and 

•  if  oi  =  o'  =  (sw,  pt,  pkt),  then  3o'  £  {o},---  ,  o'j^fl}  such 
that  the  o\  transition  is  an  In  moving  pkt  from  host  to  (sw,  pt) 
and  none  of  o' .  •  •  •  ,  o'  _  i  is  a  predecessor  of  oi,  and 

•  the  oi  transition  is  an  OUT  terminating  at  a  host. 

Intuitively,  single-packet  traces  are  end-to-end  paths  through  the 
network.  We  write  T (N)  for  the  set  of  single-packet  traces  gener¬ 
ated  by  N .  A  trace  (oi  ■  ■  ■  ok)  is  loop-free  if  o;  ^  Oj  for  all  distinct 
i  and  j  between  1  and  k.  We  consider  only  loop-free  traces,  since 
a  network  that  forwards  packets  around  a  loop  is  generally  con¬ 
sidered  to  be  misconfigured.  In  the  worst  case,  forwarding  loops 
can  cause  a  packet  storm,  wasting  bandwidth  and  degrading  perfor¬ 
mance.  Our  tool  automatically  detects/rejects  such  configurations. 

LTL  formulas.  Many  important  network  properties  can  be  under¬ 
stood  by  reasoning  about  the  traces  that  packets  can  take  through 
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the  network.  For  example,  reachability  requires  that  all  packets 
starting  at  src  eventually  reach  dst.  Temporal  logics  are  an  ex¬ 
pressive  and  well-studied  language  for  specifying  such  trace-based 
properties.  Hence,  we  use  Linear  Temporal  Logic  (LTL)  to  describe 
traces  in  our  network  model.  Let  AP  be  atomic  propositions  that 
test  the  value  of  a  switch,  port,  or  packet  field:  /,  =  n.  We  call 
elements  of  the  set  2AP  traffic  classes.  Intuitively,  each  traffic  class 
T  identifies  a  set  of  packets  that  agree  on  the  values  of  particular 
header  fields.  An  LTL  formula  p  in  negation  normal  form  (NNF) 
is  either  true,  false,  atomic  proposition  p  in  AP,  negated  propo¬ 
sition  -ip,  disjunction  pi  V  P2,  conjunction  p2  A  P2,  next  Xip, 
until  ipiU(fi2,  or  release  p\Rp2,  where  pi  and  p 2  are  LTL  for¬ 
mulas  in  NNF.  The  operators  F  and  G  can  be  defined  using  other 
connectives.  Since  (finite)  single-packet  traces  can  be  viewed  as  in¬ 
finite  sequences  of  packet  observations  where  the  final  observation 
repeats  indefinitely,  the  semantics  of  the  LTL  formulas  can  be  de¬ 
fined  in  a  standard  way  over  traces.  We  write  t  |=  p  to  indicate 
that  the  single-packet  trace  t.  satisfies  the  formula  p  and  T  |=  p 
to  indicate  that  t  \=  p  for  each  t  in  T.  Given  a  network  N  and  a 
formula  p,  we  write  N  \=  p  if  T (N)  \=  p. 

Problem  Statement.  Recall  that  our  network  model  includes 
commands  for  updating  a  single  switch,  incrementing  the  epoch, 
and  waiting  until  all  packets  in  the  preceding  epoch  have  been 
flushed  from  the  network.  At  a  high-level,  our  goal  is  to  identify 
a  sequence  of  commands  to  transition  the  network  between  config¬ 
urations  without  violating  specified  invariants.  First,  we  need  a  bit 
of  notation.  Given  a  network  N,  we  write  N[sw  <—  tbl\  for  the 
switch  update  obtained  by  updating  the  forwarding  table  for  switch 
sw  to  tbl.  We  call  N  static  if  C.cmds  is  empty.  If  static  networks 
Ni,  N„  have  the  same  traces  T(IVi)  =  T(Nn).  then  we  say  they 
are  trace-equivalent,  Ni  ~  Nn. 

Definition  2  (Network  Update).  Let  Ni  be  a  static  network.  A 
command  sequence  cmds  induces  a  sequence  N\ , ... ,  Nn  of  static 
networks  if  C\  ■  ■  ■  cn-i  are  the  update  commands  in  cmds,  and  for 
each  a  =  (sw,  tbl),  we  have  Ni[sw  <—  tbl]  ~  IVj+i. 

We  write  IVi  Nn  if  there  exists  such  a  sequence  of  static 
networks  induced  by  cmds  which  ends  with  Nn. 

We  call  N  stable  if  all  packets  in  N  are  annotated  with  the  same 
epoch.  Intuitively,  a  stable  network  is  one  with  no  in-progress  up¬ 
date,  i.e.  any  preceding  update  command  was  finalized  with  a  wait. 
Consider  the  set  of  unconstrained  single-packet  traces  generated  by 
removing  the  requirement  that  traces  start  at  an  ingress  (see  Defi¬ 
nition  8,  Appendix  A).  This  includes  T(N)  as  well  as  traces  of 
packets  initially  present  in  N.  We  call  this  'T(N),  and  note  that  for 
a  stable  network  N,  T (N)  is  equal  to  T (IV). 

Definition  3  (Update  Correctness).  Let  N  be  a  stable  static  net¬ 
work  and  let  p  be  an  LTL  formula.  The  command  sequence  cmds 
is  correct  with  respect  to  N  and  p  if  N  |=  where  N  is  obtained 
from  N  by  setting  C.  cmds  =  cmds. 

A  network  configuration  is  a  static  network  which  contains  no 
packets.  We  can  now  present  the  problem  statement. 

Definition  4  (Update  Synthesis  Problem).  Given  stable  static  net¬ 
work  N,  network  configuration  N',  and  LTL  specification  p,  con¬ 
struct  a  sequence  of  commands  cmds  such  that  (i)  N  N" 
where  N"  ~  N',  and  (ii)  cmds  is  correct  with  respect  to  p. 

3.3  Efficiently  Checking  Network  Properties 

To  facilitate  efficient  checking  of  network  properties  via  LTL  model 
checkers,  we  show  how  to  model  a  network  as  a  Kripke  structure. 

Kripke  structures.  A  Kripke  structure  is  a  tuple  (Q,Qo,S,  A), 
where  Q  is  a  finite  set  of  states,  Qo  C  Q  is  a  set  of  initial  states. 


5  C  Q  x  Q  is  a  transition  relation,  and  A  :  Q  — >  2AP  labels 
each  state  with  a  set  of  atomic  propositions  drawn  from  a  fixed  set 
AP.  A  Kripke  structure  is  complete  if  every  state  has  at  least  one 
successor.  A  state  q  £  Q  is  a  sink  state  if  for  all  states  q' ,  S(q,  q') 
implies  that  q  =  q' ,  and  we  call  a  Kripke  structure  DAG-like  if 
the  only  cycles  are  self-loops  on  sink  states.  In  this  paper,  we  will 
consider  complete  and  DAG-like  Kripke  structures.  A  trace  t  is  an 
infinite  sequence  of  states,  toti  ■  ■  .  such  that  Vi  >  0  :  5(ti,ti+i). 
Given  a  trace  t,  we  write  tl  for  the  suffix  of  t  starting  at  the  i-th 
position — i.e.,  tl  =  fifi+i  •  •  ••  Given  a  set  of  traces  T,  we  let  T! 
denote  the  set  {P  \  t  £  T}.  Given  a  state  q  of  a  Kripke  structure 
K,  let  traces  n(q)  be  the  set  of  traces  of  K  starting  from  q  and 
succk  (q)  be  the  set  of  states  defined  by  q'  £  succk  (q)  if  and  only 
if  5(q,  q').  We  will  omit  the  subscript  K  when  it  is  clear  form  the 
context.  A  Kripke  structure  K  =  (Q,Qo,5,  A)  satisfies  an  LTL 
formula  p  if  for  all  states  qo  £  Qo  we  have  that  traces  (qo)  |=  p. 

Network  Kripke  structures.  For  every  static  N,  we  can  gener¬ 
ate  a  Kripke  structure  IC(N)  containing  traces  which  correspond 
according  to  an  intuitive  trace  relation  <  (Definition  9,  10.  Ap¬ 
pendix  A).  We  currently  do  not  reason  about  packet  modification, 
so  the  Kripke  structure  has  disjoint  parts  corresponding  to  the  traf¬ 
fic  classes.  It  is  straightforward  to  enable  packet  modification,  by 
adding  transitions  between  the  parts  of  the  Kripke  structure,  but  we 
leave  this  for  future  work.  We  now  show  that  the  generated  Kripke 
structure  faithfully  encodes  the  network  semantics. 

Lemma  1  (Network  Kripke  Structure  Soundness).  Let  N  be  a 
static  network  and  K  =  IC(N)  a  network  Kripke  structure.  For 
every  single-packet  trace  t  in  T (N)  there  exists  a  trace  t'  of  K 
from  a  start  state  such  that  t  <  t' ,  and  vice  versa. 

This  means  that  checking  LTL  over  single-packet  traces  can  be 
performed  via  LTL  model-checking  of  Kripke  structures. 

Checking  network  configurations.  One  key  challenge  arises  be¬ 
cause  the  network  is  a  distributed  system.  Packets  can  “see”  an  in¬ 
consistent  configuration  (some  switches  updated,  some  not),  and 
reasoning  about  possible  interleavings  of  commands  becomes  in¬ 
tractable  in  this  context.  We  can  simplify  the  problem  if  we  ensure 
that  each  packet  traverses  at  most  one  switch  that  was  updated  after 
the  packet  entered  the  network. 

Definition  5  (Careful  Command  Sequences).  A  sequence  of  com¬ 
mands  (cmd  1  •  •  •  cmdn)  is  careful  if  every  pair  of  switch  updates 
is  separated  by  a  wait  command. 

In  the  rest  of  this  paper,  we  consider  careful  command  sequences, 
and  develop  a  sound  and  complete  algorithm  that  finds  them  ef¬ 
ficiently.  Section  4  describes  a  technique  for  removing  wait  com¬ 
mands  that  works  well  in  practice,  but  we  leave  optimal  wait  re¬ 
moval  for  future  work.  Recall  that  T (N)  denotes  the  sequence  of 
all  traces  that  a  packet  could  take  through  the  network,  regardless  of 
when  the  commands  in  N.cmds  are  executed.  This  is  a  superset  of 
the  traces  induced  by  each  static  Ni  in  a  solution  to  the  network  up¬ 
date  problem.  However,  if  cmds  is  careful,  then  each  packet  only 
encounters  a  single  configuration,  allowing  the  correctness  of  the 
sequence  to  be  reduced  to  the  correctness  of  each  Ni. 

Lemma  2  (Careful  Correctness).  Let  N  be  a  stable  network  with 
C.cmds  careful  and  let  p  be  an  LTL  formula.  If  cmds  is  careful 
and  Ni  |=  for  each  static  network  in  any  sequence  induced  by 
cmds,  then  cmds  is  correct  with  respect  to  p. 

In  Lemmas  5  and  6  (Appendix  A),  we  show  that  checking  the 
unique  sequence  of  network  configurations  induced  by  cmds  is 
equivalent  to  the  above.  Next  we  will  develop  a  sound  and  com¬ 
plete  algorithm  that  solves  the  update  synthesis  problem  for  careful 
sequences  by  checking  configurations. 
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Procedure  ORDERUPDATE(iVj ,  Nf,cp) 

Input:  Initial  static  network  N{,  final  static  configuration  Nf,  formula  ip. 
Output:  update  sequence  L,  or  error  e  if  no  update  sequence  exists 
1:  W  <—  false  >  Formula  encoding  wrong  configurations. 

2:  V  <—  false  >  Formula  encoding  visited  configurations. 

3:  (ok,  L )  <—  DFSforOrder (Ni,JC(Ni),  _L,  p,  Ao) 

4:  if  ok  then  return  L 

5:  else  return  e  >  Failure — no  update  exists. 

Procedure  DF SforOrder(JV ,K ,s,(p,X) 

Input:  Static  network  N  and  Kripke  structure  K,  next  switch  to  update  s, 
formula  tp,  and  labeling  A. 

Output:  Boolean  ok  if  a  correct  update  exists;  correct  update  sequence  L 
6:  if  TV  |=  V  V  W  then  return  (false,  []) 

7:  if  s  =  _L  then  (ok,  cex,  A)  <—  modelCheck(K ,  p) 

8:  else 

9:  (TV ,  K,  S)  <—  swUpdate(N ,  s) 

10:  (ok,  cex,  A)  <—  incrModelCheck(K,p,  S,X) 

11:  V  <—  V  V  makeFormula(N) 

12:  if  — >o/c  then 

13:  W  <—  W  V  makeFormula(cex) 

14:  return  (false,  []) 

15:  if  TV  =  Nf  then  return  (true,  [s]) 

16:  for  s'  S  possibleUpdates(N)  do 

17:  (ok,  L)  <-  DFSforOrder(TV,  K,  s',  p,  A) 

18:  if  ok  then  return  (true,  (upd  s')  ::  wait  ::  L ) 

19:  return  (false,  []) 

Figure  4:  ORDERUPDATE  Algorithm. 

4.  Update  Synthesis  Algorithm 

This  section  presents  a  synthesis  algorithm  that  searches  through 
the  space  of  possible  solutions,  using  counterexamples  to  detect 
wrong  configurations  and  exploiting  several  optimizations. 

4.1  Algorithm  Description 

ORDERUPDATE  (Figure  4)  returns  a  simple  sequence  of  updates 
(one  in  which  each  switch  appears  at  most  once),  or  fails  if  no  such 
sequence  exists.  Note  that  we  could  broaden  our  simple  definition, 
e.g.  k-simple,  where  each  switch  appears  at  most  k  times,  but  we 
have  found  the  above  restriction  to  work  well  in  practice.  The  core 
procedure  is  DFSforOrder,  which  manages  the  search  and  in¬ 
vokes  the  model  checker  (we  use  DFS  because  we  expect  common 
properties/configurations  to  admit  many  update  sequences).  It  at¬ 
tempts  to  add  a  switch  s  to  the  current  update  sequence,  yielding  a 
new  network  configuration.  We  maintain  two  formulas,  V  and  W, 
tracking  the  set  of  configurations  that  have  been  visited  so  far,  and 
the  set  of  configurations  excluded  by  counterexamples. 

To  check  whether  all  packet  traces  in  this  configuration  satisfy 
the  LTL  property  p,  we  use  our  (incremental)  model  checking 
algorithm  (discussed  in  Section  5).  First,  we  call  a  full  check  of  the 
model  (line  7).  The  model  checker  labels  the  Kripke  structure  nodes 
with  information  about  what  formulas  hold  for  paths  starting  at  that 
state.  The  labeling  (stored  in  A)  is  then  re-used  in  the  subsequent 
model  checking  calls  for  related  Kripke  structures  (line  10).  The 
parameters  passed  in  the  incremental  model  checking  call  are: 
updated  Kripke  structure  K,  specification  p,  set  of  nodes  S  in 
K  whose  transition  function  has  changed  by  the  update  of  the 
switch  s,  and  correct  labeling  A  of  the  Kripke  structure  before  the 
update.  Note  that  before  the  initial  model  checking,  we  convert 
the  network  configuration  TV  to  a  Kripke  structure  K.  The  update 
of  K  is  performed  by  a  function  swUpdate  that  returns  a  triple 
(TV',  S,  K'),  where  TV'  is  the  new  static  network.  K'  is  the  updated 
Kripke  structure  obtained  as  IC(N'),  and  S  is  the  set  of  nodes  that 
have  different  outgoing  transitions  in  K'. 

If  the  model  checker  returns  true,  then  TV  is  safe  and  the  search 
proceeds  recursively,  after  adding  (upd  s')  to  the  current  sequence 


of  commands.  If  the  model  checker  returns  false,  the  search  back¬ 
tracks,  using  the  counterexample-learning  approach  below. 

4.2  Optimizations 

We  now  present  optimizations  improving  synthesis  (pruning  with 
counterexamples,  early  search  termination),  and  improving  effi¬ 
ciency  of  synthesized  updates  (wait  removal). 

A.  Counterexamples.  Counterexample-based  pruning  learns  net¬ 
work  configurations  that  do  not  satisfy  the  specification  to  avoid 
making  future  model  checking  calls  that  are  certain  to  fail.  The 
function  makeFormula(cex)  (Line  13)  returns  a  formula  repre¬ 
senting  the  set  of  switches  that  occurred  in  the  counterexample 
trace  cex,  with  flags  indicating  whether  each  switch  was  updated. 
This  allows  equivalent  future  configurations  to  be  eliminated  with¬ 
out  invoking  the  model  checker.  Recall  the  red-green  example  in 
Section  2  and  suppose  that  we  update  A1  and  then  C2.  At  the  inter¬ 
mediate  configuration  obtained  by  updating  just  Al,  packets  will  be 
dropped  at  C2,  and  the  specification  (H1-H3  connectivity)  will  not 
be  satisfied.  The  formula  for  the  unsafe  set  of  configurations  that 
have  Al  updated  and  C2  not  updated  will  be  added  to  W .  In  prac¬ 
tice,  many  counterexamples  are  small  compared  to  network  size, 
and  this  greatly  prunes  the  search  space. 

B.  Early  search  termination.  The  early  search  termination  op¬ 
timization  speeds  up  termination  of  the  search  when  no  (switch- 
granularity)  update  sequence  is  possible.  Recall  how  we  use  coun¬ 
terexamples  to  prune  configurations.  With  similar  reasoning,  we 
can  use  counterexamples  for  pruning  possible  sequences  of  up¬ 
dates.  Consider  a  counterexample  trace  which  involves  three  nodes 
A,  B,  C,  with  A  updated,  B  updated,  and  C  not  updated.  This  can 
be  seen  as  requiring  that  C  must  be  updated  before  A,  or  C  must 
be  updated  before  B.  Early  search  termination  involves  collecting 
such  constraints  on  possible  updates,  and  terminating  if  these  con¬ 
straints  taken  together  form  a  contradiction.  In  our  tool,  this  is  done 
efficiently  using  an  (incremental)  SAT  solver.  If  the  solver  deter¬ 
mines  that  no  update  sequence  is  possible,  the  search  terminates. 
For  simplicity,  early  search  termination  is  not  shown  in  Figure  4. 

C.  Wait  removal.  This  heuristic  eliminates  waits  that  are  un¬ 
necessary  for  correctness.  Consider  an  update  sequence  L  = 
cmdocmdi  ■  ■  ■  cmdn,  and  consider  some  switch  update  cmdk  = 
(upd  s).  In  the  configuration  resulting  from  executing  the  sequence 
cmdocmdi  •  •  •  cm4-i.  if  the  switch  s  cannot  possibly  receive 
a  packet  which  passed  through  some  switch  so  before  an  update 
cmdj  —  (upd  so)  where  j  <  k,  then  we  can  update  s  without  wait¬ 
ing.  Thus,  we  can  remove  some  unnecessary  waits  if  we  can  main¬ 
tain  reachability-between-switches  information  during  the  update. 
Wait  removal  is  not  shown  in  Figure  4,  but  in  our  tool,  it  operates  as 
a  post-processing  pass  once  an  update  sequence  is  found.  In  prac¬ 
tice.  this  removes  a  majority  of  unnecessary  waits  (see  §  6). 

4.3  Formal  Properties 

The  following  two  theorems  show  that  our  algorithm  is  sound  for 
careful  updates,  and  complete  if  we  limit  our  search  to  simple 
update  sequences  (see  Appendix  B  for  proofs). 

Theorem  1  (Soundness).  Given  initial  network  Ni,  final  configu¬ 
ration  Nf,  and  LTL  formula  p,  if  ORDERUPDATE  returns  a  com¬ 
mand  sequence  cmds,  then  Ni  c—t  N'  s.t.  N'  ~  TV/,  and  cmds 
is  correct  with  respect  to  p  and  Ni. 

Theorem  2  (Completeness).  Given  initial  network  N,  final  con¬ 
figuration  Nf,  and  specification  p,  if  there  exists  a  simple,  careful 

sequence  cmds  with  Ni  TV'  s.t.  TV'  ~  TV/,  then  OrderUp- 
DATE  returns  one  such  sequence. 
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5.  Incremental  Model  Checking 

We  now  present  an  incremental  algorithm  for  model  checking 
Kripke  structures.  This  algorithm  is  central  to  our  synthesis  tool, 
which  invokes  the  model  checker  on  many  closely  related  struc¬ 
tures.  The  algorithm  makes  use  of  the  fact  that  the  only  cycles  in  the 
Kripke  structure  are  self-loops  on  sink  nodes — something  that  is 
true  of  structures  encoding  loop-free  network  configurations — and 
re-labels  the  states  of  a  previously-labeled  Kripke  structure  with  the 
(possibly  different)  formulas  that  hold  after  an  update. 


Holdso  ( q,p ) 
Holdso  ( q ,  ~>p) 
Holdso  (q,  0  1  A  02) 
Holdso  (q,  (pi  V  02 ) 
Holdso  (q,Xcp) 
Holdso (q,  (pi  U  <p2) 
Holdso  (q,  (pi  R  </>2) 


q\=p 

qV=p 

Holdso(q,(pi)  A  Holdso(q,  (p2) 
Holdso{q,(pi)  V  Holdso{q,(p2) 
Holdso  (q,(p) 

Holds0  (q,(p2) 

Holdso{q,(pi)  V  Holdso{q,(p2) 


Figure  5:  The  Holdso  function 


5.1  State  Labeling 

We  begin  with  an  algorithm  for  labeling  states  of  a  Kripke  struc¬ 
ture  with  sets  of  formulas,  following  the  approach  of  [39]  (WVS) 
and  [37],  The  WVS  algorithm  translates  an  LTL  formula  < p  into  a 
local  automaton  and  an  eventuality  automaton.  The  local  automa¬ 
ton  checks  consistency  between  a  state  and  its  predecessor,  and 
handles  labeling  of  all  formulas  except  pi  U  p2,  which  is  checked 
by  the  eventuality  automaton.  The  two  automata  are  composed  into 
a  single  Biichi  automaton  whose  states  correspond  to  subsets  of  the 
set  of  subformulas  of  p  and  their  negations.  Hence,  we  label  each 
Kripke  state  by  a  set  L  of  sets  of  formulas  such  that  if  a  state  q  is 
labeled  by  L,  then  for  each  set  of  formulas  S  in  L,  there  exists  a 
trace  t.  starting  from  q  satisfying  all  the  formulas  in  S. 

We  now  describe  state  labeling  precisely.  Let  p  be  an  LTL 
formula  in  NNF.  The  extended  closure  of  p,  written  ecl(p),  is  the 
set  of  all  subformulas  of  p  and  their  negations: 

•  true  £  ecl(p) 

•  p  €  ecl(p ) 

•  If  ip  £  ecl(p),  then  -up  £  ecl(p) 

(we  identify  ip  with  -i-i ip,  for  all  ip). 

•  If  ¥>i  V  <^2  G  ecl(p),  then  pi  £  ecl(p)  and  p2  £  ecl(p). 

•  If  V5i  A  G  ecl(p),  then  pi  £  ecl(p)  and  p2  G  ecl(p). 

•  If.Yipi  £  ecl(p),  then  pi  £  ecl(p). 

•  If  pi  U  p2  £  ecl(p),  then  pi  £  ecl(p)  and  p2  G  ecl(p) 

•  If  V2i  i?  ^2  G  ecl(p),  then  pi  £  ecl(p)  and  p2  £  ecl(p). 

A  subset  M  C  ecl(p)  of  the  extended  closure  is  said  to  be 
maximally  consistent  if  it  contains  true  and  is  simultaneously 
closed  and  consistent  under  boolean  operations: 

•  true  £  M 

•  ip  £  M  iff  -i ip  pp  M  (we  identify  ip  with  -i-> ip,  for  all  ip) 

•  pi's/  P2  £  M  iff  (pi  £  M  or  p2  G  M) 

•  pi  A  p2  £  M  iff  (pi  £  M  and  p2  £  M) 

Likewise,  the  relation  follows  (Mi ,  M2)  captures  the  notion  of  suc¬ 
cessor  induced  by  LTL’s  temporal  operators,  lifted  to  maximally- 
consistent  sets.  We  say  follows(Mi,  M2)  holds  if  and  only  if  all  of 
the  following  hold: 

•  X  pi  £  Mi  iff  pi  £  M2 

•  V2i  ?7  ^2  G  Mi  iff  (yp2  £  Mi  V  (pi  £  Mi  A  piU  P2  £  M2)) 

•  Pi  Rp2  £  Mi  iff  (ypi  £  Mi  V  (P2  £  Mi  Api  Rp2  £  M2)) 
Given  a  trace  t  and  a  maximally-consistent  set  M,  we  write  t  |=  M 
if  and  only  if  for  all  ip  £  M,  we  have  t  \=  ip. 

For  the  rest  of  this  section,  we  fix  a  Kripke  structure  K  = 
(Q,  Q 0,  S,  A),  a  state  q  in  Q,  an  LTL  formula  p  in  NNF,  and  a 
maximally-consistent  set  M  C  ecl(p). 

To  compute  the  label  of  a  state  q,  there  are  two  cases  depending 
on  whether  it  is  a  sink  state  or  a  non-sink  state.  If  q  is  a  sink  state, 
the  function  HoldsSink(q,  M)  computes  a  predicate  that  is  true  if 
and  only  if,  for  all  ip  £  M  and  the  unique  trace  t  starting  from  q, 
we  have  t\=ip.  More  formally.  HoldsSink(q ,  M)  is  defined  to  be 
(\/ip  £  M  :  Holdso(q,ip)),  where  Holdso  is  defined  as  in  Figure 
5.  The  function  Holdso  computes  a  predicate  that  is  true  if  and 
only  if  ip  holds  at  q.  For  example,  Holdso  (q,  (pi  U  02)  is  defined 
as  Holdso  (q,  02)  because  the  only  transition  from  q  is  a  self-loop. 


For  the  second  case,  suppose  q  is  a  non-sink  state.  If  we  are 
given  a  labeling  for  succK(q)  (the  successors  of  the  node  q),  we 
can  extend  it  to  a  labeling  for  q.  Let  V  C  Q  be  a  set  of  vertices.  A 
function  labGrK  is  a  correct  labeling  of  K  with  respect  to  p  and 
V  if  for  every  v  £  V,  it  returns  a  set  L  of  maximally  consistent 
sets  such  that  (a)  M  £  L  if  and  only  if  M  C  ecl(p),  and  (b) 
there  exists  a  trace  t  in  traces(v)  such  that  t  |=  M.  Suppose  that 
labGrK  is  a  correct  labeling  of  K  with  respect  to  p  and  succK(q). 
The  function  Holds K(q,  M,  labGrK)  computes  a  predicate  that  is 
true  if  and  only  if  there  exists  a  trace  t  in  traces K(q)  with  t  |=  M. 
Formally,  HoldsK(q,  M,  labGrK)  is  defined  as  (A (q)  =  (AP  (T 
M))  A  3 q'  £  succK(q),  M'  £  labGrK(q')  :  follows (M,  M'). 

The  following  captures  the  correctness  of  labeling: 

Lemma  3.  First,  H olds  Sink  (q,  M)  3 t  £  traces(q)  :  t  |=  M 
for  sink  states  q.  Second,  if  labGrK  is  a  correct  labeling  with 
respect  to  p  and  succK(q),  then  Holds K(q,M,  labGrK)  <=^ 
3 1  £  traces K(q)  :t\=M. 

Finally,  we  define  labelNodeK(p,  q,  labGrK),  which  com¬ 
putes  a  label  L  for  q  such  that  M  £  L  if  and  only  if  there  exists 
a  trace  t  £  tracesK(q)  such  that  t  |=  M  for  all  M  C  ecl(p). 
We  assume  that  labGrK  is  a  correct  labeling  of  K  with  respect  to 
p  and  succ(q).  For  sink  states,  labelNodeK(p,  q,  labGrK)  returns 
{M  |  M  £  ecl(p)  A  HoldsSink(q,  M)},  while  for  non-sink  states 
it  returns  {M  \  M  £  ecl(p)  A  HoldsK(q,M,  labGrK)}- 

5.2  Incremental  algorithm 

To  incrementally  model  check  a  modified  Kripke  structure,  we 
must  re-label  its  states  with  the  formulas  that  hold  after  the  update. 

Consider  two  Kripke  structures  K  =  ( Q ,  Qq,  5,  A)  and  K'  = 
(Q',Qo,S',X'),  such  that  Q 0  =  Q'0.  Furthermore,  assume  that 
Q  =  Q',  and  there  is  a  set  U  C  Q  such  that  S  and  S'  differ  only  on 
nodes  in  U.  We  call  such  a  triple  ( K ,  K' ,  U)  an  update  of  K. 

An  update  (K.  K1  ,U)  might  add  or  remove  edges  connected 
to  a  (small)  set  of  nodes,  corresponding  to  a  change  in  the  rules 
on  a  switch.  Suppose  that  labGrK  is  a  correct  labeling  of  K  with 
respect  to  p  and  Q.  The  incremental  model  checking  problem  is  de¬ 
fined  as  follows:  we  are  given  an  update  ( K ,  A'',  U),  and  labGrK, 
and  we  want  to  know  whether  K'  satisfies  p.  The  naive  approach  is 
to  model  check  K'  without  using  the  labeling  labGrK-  We  call  this 
the  monolithic  approach.  In  contrast,  the  incremental  approach  uses 
labGrK  (and  thus  intuitively  re-uses  the  results  of  model  checking 
K  to  efficiently  verify  K'). 

Example.  Consider  the  left  side  of  Figure  6,  with  H  the  only 
initial  state.  Suppose  that  the  update  modifies  J,  and  the  S’  relation 
applied  to  J  only  contains  the  pair  (J,N),  and  consider  labeling 
the  structure  with  formulas  F  a,  F  b,  and  F  a  \J  F  b.  To  simplify 
the  example,  we  label  a  node  by  all  those  formulas  which  hold  for 
at  least  one  path  starting  from  the  node  (note  that  in  the  algorithm, 
a  node  is  labeled  by  a  set  of  sets  of  formulas,  rather  than  a  set  of 
formulas).  We  will  have  that  all  the  nodes  are  labeled  by  A  a  V  F  b, 
and  in  addition  the  nodes  K,  I,  H,  M,  J  contain  label  F  a,  and  the 
nodes  L,  I,  H,  N  contain  F  b.  Now  we  want  to  relabel  the  structure 
after  the  update  (right-hand  side).  Given  that  the  update  changes 
only  node  J.  the  labeling  can  only  change  for  J  and  its  ancestors. 
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Figure  6:  Incremental  labeling — Initial  (left).  Final  (right) 


We  therefore  start  labeling  node  J,  and  find  that  it  will  now  be 
labeled  with  F  b  instead  of  F  a.  Labeling  proceeds  to  H,  whose 
label  does  not  change  (still  labeled  by  all  of  F  a,  F  b,  F  a  V  F  b). 
The  labeling  process  could  then  stop,  even  if  H  has  ancestors. 


6.  Implementation  and  Experiments 

We  have  built  a  prototype  tool  that  implements  the  algorithms  de¬ 
scribed  in  this  paper.  It  consists  of  7K  lines  of  OCaml  code.  The 
system  works  by  building  a  Kripke  structure  (§3)  and  then  repeat¬ 
edly  interacting  with  a  model  checker  to  synthesize  an  update.  We 
currently  provide  four  checker  backends:  Incremental  uses  incre¬ 
mental  relabeling  to  check  and  recheck  formulas,  Batch  re-labels 
the  entire  graph  on  each  call,  NuSMV  queries  a  state-of-the-art 
symbolic  model  checker  in  batch  mode,  and  NetPlumber  queries 
an  incremental  network  model  checker  [19].  All  tools  except  Net- 
Plumber  provide  counterexample  traces,  so  our  system  learns  from 
counterexamples  whenever  possible  (§4). 


Re-labeling  states.  Let  ancestors k(V)  be  the  ancestors  of  V 
in  K — i.e.,  a  set  of  vertices  s.t.  ancestors k(V)  C  Q  and  q  £ 
ancestors k(V),  if  some  node  v  £  V  is  reachable  from  q.  To  de¬ 
fine  incremental  model  checking  for  p,  we  need  a  function  accept¬ 
ing  a  property  p,  set  of  vertices  V,  labeling  labGrK  that  is  correct 
for  K  with  respect  to  p  and  Q  \  ancestors k(V),  and  returns  a 
correct  labeling  of  I\  with  respect  to  p  and  Q.  This  function  is: 


relblic(p,  labGrK,  V) 


labGr  K  if  V  =  0 

relblnip,  labGr'K,V')  otherwise 


where  labGr'K(v )  is  labelN ode k{p,v,  labCra)  if  v  £  V,  and 
it  is  labGr k(v)  if  v  V .  The  set  V1  is  {q  \  3v  £  V  :  v  £ 
succK{q)}- 


Theorem  3.  Let  V  C  Q  be  a  set  of  vertices  and  labGr k  a 
correct  labeling  with  respect  to  p  and  Q  \  ancestors k(V).  Then 
relblx(p ,  labGrK ,  V)  is  a  correct  labeling  w.r.t.  p  and  Q. 


Given  a  labeling  that  is  correct  with  respect  to  p  and  Q,  it 
is  easy  to  check  whether  p  is  true  for  all  the  traces  starting  in 
the  initial  states:  the  predicate  checkInitStatesK{labGrK,p)  is 
defined  as  Vqo  £  Qo,M  £  labGr K^qo)  ■  P  £  M.  Next, 
let  Qf  be  the  set  of  all  sink  states  of  K.  Then  ancestors k (Q /) 
is  the  set  Q  of  all  states  K.  Therefore,  for  any  initial  labeling 
labGr°K,  relbl(p,  labGr°K ,  Qf)  is  a  correct  labeling  with  respect 
to  p  and  Q.  The  function  modelCheckK{p)  is  defined  to  be  equal 
to  checkInitStatesK{relblK{p,  labGr°K,Q  f),p),  where  we  can 
set  labGr^  to  be  the  empty  labeling  Au.0. 

We  now  define  our  incremental  model  checking  function.  Let 
( K ,  K' ,  U)  be  an  update,  and  labGrK  a  previously-computed  cor¬ 
rect  labeling  of  I\  with  respect  to  p  and  Q ,  where  Q  is  the  set 
of  states  of  K .  The  function  incrModelCheck(K ,  p,  U,  labGrK) 
is  defined  as  checklnitStates k1  {relbl k1  (p,  labGrK,  U),  p).  The 
following  shows  the  correctness  of  our  model  checking  functions 
(proof  of  this  and  the  previous  theorem  are  in  Appendix  C). 


Corollary  1.  First,  modelCheckK(p)  =  true  4=4*  K  \= 
p.  Second,  for  ( K,K',U )  and  labGrK  as  above,  we  have 
incrModelCheck(K,  p,U,  labGrK)  =  true  4=4  K  |=  p. 


The  runtime  complexity  of  the  modelCheckK  function  is 
0(1*1  x  2^).  The  runtime  complexity  of  the  incrModelCheck 
function  is  0(\ ancestors  k(U)\  x  2^),  where  U  is  the  set  of  nodes 
being  updated. 


Counterexamples.  This  incremental  algorithm  can  generate 
counterexamples  in  cases  where  the  formula  does  not  hold.  A  for¬ 
mula  -i p  does  not  hold  if  an  initial  state  is  labeled  by  L,  such  that 
there  exists  a  set  M  £  L,  such  that  ->p  £  M.  Examining  the  def¬ 
inition  of  labelNodeK ,  we  find  that  in  order  to  add  a  set  M  to  the 
label  L  of  a  node  q,  there  is  a  set  M1  in  the  label  of  one  a  child  q'  of 
q  that  explains  why  M  is  in  L.  The  first  node  of  the  counterexample 
trace  starting  from  q  is  one  such  child  q' . 


Experiments.  To  evaluate  performance,  we  generated  configura¬ 
tions  for  a  variety  of  real-world  topologies  and  ran  experiments 
in  which  we  measured  the  amount  of  time  needed  to  synthesize 
an  update  (or  discover  that  no  order  update  exists).  These  experi¬ 
ments  were  designed  to  answer  two  key  questions:  (1)  how  the  per¬ 
formance  of  our  Incremental  checker  compares  to  state-of-the-art 
tools  (NuSMV  and  NetPlumber),  and  (2)  whether  our  synthesizer 
scales  to  large  topologies.  We  used  the  Topology  Zoo  [21]  dataset, 
which  consists  of  261  actual  wide-area  topologies,  as  well  as  syn¬ 
thetically  constructed  Small-World  [29]  and  FatTree  [1]  topologies. 
We  ran  the  experiments  on  a  64-bit  Ubuntu  machine  with  20GB 
RAM  and  a  quad-core  Intel  i5-4570  CPU  (3.2  GHz)  and  imposed 
a  10-minute  timeout  for  each  run.  We  ignored  runs  in  which  the 
solver  died  due  to  an  out-of-memory  error  or  timeout — these  are 
infrequent  (less  than  8%  of  the  996  runs  for  Figure  7),  and  our  In¬ 
cremental  solver  only  died  in  instances  where  other  solvers  did  too. 

Configurations  and  properties.  A  recent  paper  [23]  surveyed 
data-center  operators  to  discover  common  update  scenarios,  which 
mostly  involve  taking  switches  on/off-line  and  migrating  traffic  be¬ 
tween  switches/hosts.  We  designed  experiments  around  a  similar 
scenario.  To  create  configurations,  we  connected  random  pairs  of 
nodes  (s,  d)  via  disjoint  initial/final  paths  W,,  Wf,  forming  a  “dia¬ 
mond”,  and  asserted  one  of  the  following  properties  for  each  pair: 

•  Reachability:  traffic  from  a  given  source  must  reach  a  certain 
destination:  (port  =  s)  =4  F  (port  =  d) 

•  Waypointing:  traffic  must  traverse  a  waypoint  w: 

(port=s)  =4*  ((ported)  U  ((port=tn)  A  F  ( port=d ))) 

•  Sendee  chaining:  traffic  must  waypoint  through  several  inter¬ 
mediate  nodes:  (port  =  s)  =4*  way(W,  d),  where 

way([\,d)  =  F  (port  =  d) 

way{wi  ::  W,d)  =  (( AWk€W  Port^wfe  A  ported) 

U  ((port  =  Wi)  A  way(W,  d))) . 

Incremental  vs.  NuSMV/Batch.  Figure  7  (a-c)  compares  the  per¬ 
formance  of  Incremental  and  NuSMV  backends  for  the  reachability 
property.  Of  the  247  Topology  Zoo  inputs  that  completed  success¬ 
fully,  our  tool  solved  all  of  them  faster.  The  measured  speedups 
were  large,  with  a  geometric  mean  of  447. 23x.  For  the  24  FatTree 
examples,  the  mean  speedup  was  465. 03x,  and  for  the  25  Small- 
World  examples,  the  mean  speedup  was  4484. 73x.  We  also  com¬ 
pared  the  Incremental  and  Batch  solvers  on  the  same  inputs.  Incre¬ 
mental  performs  better  on  almost  all  examples,  with  mean  speedup 
of  4.26x,  5.27x,  11.74x  on  the  datasets  shown  in  Figure  7(a-c)  and 
maximum  runtimes  of  0.36s,  2.80s,  and  0.92s  respectively.  The 
maximum  runtimes  for  Batch  were  6.71s,  39.75s,  and  12.50s. 

Incremental  vs.  NetPlumber.  We  also  measured  the  performance 
of  Incremental  versus  the  network  property  checker  NetPlumber 
(Figure  7(d-f)).  Note  that  NetPlumber  uses  rule-granularity  for 
updates,  so  we  enabled  this  mode  in  our  tool  for  these  experiments. 
For  the  three  datasets,  our  checker  is  faster  on  all  experiments,  with 
mean  speedups  of  (6.41x,  4.90x,  17.19x).  NetPlumber  does  not 
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ists  (i.e.  algorithm  reports  “impossible”),  and  (i)  Scalability  of  fine-grained 
(rule-granularity)  approach  for  solving  switch-impossible  examples  in  (h). 


report  counterexamples,  putting  it  at  a  disadvantage  in  this  end- 
to-end  comparison,  so  we  also  measured  total  Incremental  versus 
NetPlumber  runtime  on  the  same  set  of  model-checking  questions 
posed  by  Incremental  for  the  Small-World  example.  Our  tool  is  still 
faster  on  all  instances,  with  a  mean  speedup  of  2.74x. 

Scalability.  To  quantify  our  tool’s  scalability,  we  constructed 
Small  World  topologies  with  up  to  1500  switches,  and  ran  experi¬ 
ments  with  large  diamond  updates — the  largest  has  1015  switches 
updating.  The  results  appear  in  Figure  8(g).  The  maximum  synthe¬ 
sis  times  for  the  three  properties  were  129.04s,  30.11s,  and  0.85s, 
which  shows  that  our  tool  scales  to  problems  of  realistic  size. 

Infeasible  Updates.  We  also  considered  examples  for  which 
there  is  no  switch-granular  update.  Figure  8(h)  shows  the  results 
of  experiments  where  we  generated  a  second  diamond  atop  the  first 
one,  requiring  it  to  route  traffic  in  the  opposite  direction.  Using 
switch-granularity,  the  inputs  are  reported  as  unsolvable  in  maxi¬ 
mum  time  153.48s,  33.48s,  and  0.69s.  Using  rule-granularity,  these 
inputs  are  solved  successfully  for  up  to  1000  switches  with  maxi¬ 
mum  times  of  776.13s,  512.84s,  and  82.00s  (see  Figure  8(i)). 

Waits.  We  also  separately  measured  the  time  needed  to  run  the 
wait-removal  heuristic  for  the  Figure  8  experiments.  For  (g),  the 
maximum  wait-removal  runtime  was  0.89s,  resulting  in  2  needed 
waits  for  each  instance.  For  (i),  the  maximum  wait-removal  runtime 
was  103.87s,  resulting  in  about  2.6  waits  on  average  (with  a  maxi¬ 
mum  of  4).  For  the  largest  problems  in  (g)  and  (i),  this  corresponds 
to  removal  of  1397/1399  and  55823/55826  waits  (about  99.9%). 


7.  Related  Work 

This  paper  extends  preliminary  work  reported  in  a  workshop  pa¬ 
per  [30],  We  present  a  more  precise  and  realistic  network  model, 
and  replace  expensive  calls  to  an  external  model  checker  with  calls 
to  a  new  built-in  incremental  network  model  checker.  We  extend 
the  DFS  search  procedure  with  optimizations  and  heuristics  that 
improve  performance  dramatically.  Finally,  we  evaluate  our  tool  on 
a  comprehensive  set  of  benchmarks  with  real-world  topologies. 

Synthesis  of  concurrent  programs.  There  is  much  previous  work 
on  synthesis  for  concurrent  programs  [12,  35,  38].  In  particular, 
work  by  Solar-Lezama  et  al.  [35]  and  Vechev  et  al.  [38]  synthesizes 
sequences  of  instructions.  However,  traditional  synthesis  and  syn¬ 
thesis  for  networking  are  quite  different.  First,  traditional  synthesis 
is  a  game  against  the  environment  which  (in  the  concurrent  pro¬ 
gramming  case)  provides  inputs  and  schedules  threads;  in  contrast, 
our  synthesis  problem  involves  reachability  on  the  space  of  config¬ 
urations.  Second,  our  space  of  configurations  is  very  rich,  meaning 
that  checking  configurations  is  itself  a  model  checking  problem. 

Network  updates.  There  are  many  protocol-  and  property- 
specific  algorithms  for  implementing  network  updates,  e.g.  avoid¬ 
ing  packet/bandwidth  loss  during  planned  maintenance  to  BGP  [10, 
32],  Other  work  avoids  routing  loops  and  blackholes  during  IGP 
migration  [36].  Work  on  network  updates  in  SDN  proposed  the 
notion  of  consistent  updates  and  several  implementation  mech¬ 
anisms,  including  two-phase  updates  [33].  Other  work  explores 
propagating  updates  incrementally,  reducing  the  space  overhead  on 
switches  [17],  As  mentioned  in  Section  2,  recent  work  proposes  or¬ 
dering  updates  for  specific  properties  [15],  whereas  we  can  handle 
combinations  and  variants  of  these  properties.  Furthermore,  SWAN 
and  zUpdate  add  support  for  bandwidth  guarantees  [13,  23].  Zhou 
et  al.  [40]  consider  customizable  trace  properties,  and  propose  a  dy¬ 
namic  algorithm  to  find  order  updates.  This  solution  can  take  into 
account  unpredictable  delays  caused  by  switch  updates.  However, 
it  may  not  always  find  a  solution,  even  if  one  exists.  In  contrast,  we 
obtain  a  completeness  guarantee  for  our  static  algorithm.  Ludwig 
et  al.  [24]  consider  ordering  updates  for  waypointing  properties. 

Model  checking.  Model  checking  has  been  used  for  network  ver¬ 
ification  [2,  18,  20,  26,  27],  The  closest  to  our  work  is  the  incre¬ 
mental  checker  NetPlumber  [19],  Surface-level  differences  include 
the  specification  languages  (LTL  vs.  regular  expressions),  and  Net- 
Plumber’s  lack  of  counterexample  output.  The  main  difference  is 
incrementality:  Netplumber  restricts  checking  to  “probe  nodes,” 
keeping  track  of  “header-space”  reachability  information  for  those 
nodes,  and  then  performing  property  queries  based  on  this.  In  con¬ 
trast,  we  look  at  the  property ,  keeping  track  of  portions  of  the 
property  holding  at  each  node,  which  keeps  incremental  recheck- 
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ing  times  low.  The  empirical  comparison  (Section  6)  showed  better 
performance  of  our  tool  as  a  back-end  for  synthesis. 

Incremental  model  checking  has  been  studied  previously,  with 

[34]  presenting  the  first  incremental  model  checking  algorithm,  for 
alternation-free  /r-calculus.  We  consider  LTL  properties  and  spe¬ 
cialize  our  algorithm  to  exploit  the  no-forwarding-loops  assump¬ 
tion.  The  paper  [7]  introduced  an  incremental  algorithm,  but  it  is 
specific  to  the  type  of  partial  results  produced  by  IC3  [5]. 

8.  Conclusion 

We  present  a  practical  tool  for  automatically  synthesizing  cor¬ 
rect  network  update  sequences  from  formal  specifications.  We  dis¬ 
cuss  an  efficient  incremental  model  checker  that  performs  orders 
of  magnitude  better  than  state-of-the-art  monolithic  tools.  Exper¬ 
iments  on  real-world  topologies  demonstrate  the  effectiveness  of 
our  approach  for  synthesis.  In  future  work,  we  plan  to  explore  both 
extensions  to  deal  with  network  failures  and  bandwidth  constraints, 
and  deeper  foundations  of  techniques  for  network  updates. 
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A.  Network  Model  Auxiliary  Definitions 

We  first  define  what  it  means  for  a  table  to  be  active,  i.e.  the 

controller  contains  an  update  that  will  eventually  produce  that  table. 

Definition  6  (Active  Forwarding  Table).  Let  N  be  a  network.  The 

forwarding  table  tbl  is  active  in  the  epoch  epfor  the  switch  sw  if 

1.  ep  =  0  and  tbl  is  the  initial  table  of  sw  in  N,  or 

2.  ep  >  0  and  either  (a)  if  there  exists  a  command  (sw' ,  tbl')  £ 
C.cmds  such  that  sw  =  sw'  and  the  number  of  wait  com¬ 
mands  preceding  (sw,  tbl)  in  C.cmds  is  ep,  then  tbl  =  tbl',  or 
(b)  if  there  does  not  exist  such  a  command,  then  tbl  is  the  table 
active  for  the  switch  sw  in  epoch  ep  —  1. 

Next  we  define  what  it  means  for  an  observation  o'  to  succeed  o. 


Definition  7  (Successor  Observation).  Let  N  be  a  network  and  let 
o  =  (sw,pt,pkt)  and  o'  =  (sw' ,  pt' ,  pkt')  be  obsemations.  The 

ep 

obsen’ation  o'  is  a  successor  of  o  in  ep,  written  o  C  o',  if  either: 

•  there  exists  a  switch  Si  and  link  Lj  such  that  Si.sw  =  sw  and 
Si. tbl  is  active  in  ep  and  Lj.loc  =  (sw,ptj)  and  Lj.loc'  = 
(sw',pt')  and  (ptj,pkt')  £  \Si.tbl\(pt,pkt),  or 

•  there  exists  a  switch  Si,  a  link  Lj,  and  a  host  h  such  that 
Si.sw  =  sw  and  Si. tbl  is  active  in  ep  and  Lj.loc  =  (sw,  pt') 
and  Lj.loc'  =  hand  (pt',pkt')  £  \Si.tbl\(pt,pkt). 

ep 

Intuitively  o  □  o'  if  the  packet  in  o  could  have  directly  produced 
the  packet  in  o'  in  ep  by  being  processed  on  some  switch.  The  two 
cases  correspond  to  an  internal  and  egress  processing  steps. 


Definition  8  (Unconstrained  Single-Packet  Trace).  Let  N  be  a 
network.  The  sequence  (oi  ■■■  of)  is  a  unconstrained  single-packet 

trace  of  N  if  N  — 4  . . .  — Nk  such  that  (oi  •  •  •  of  is  a 
subsequence  of  (d^  •  ■  ■  o'k)  for  which 


every  observation  is  a  successor  of  the  preceding  obsen’ation 
in  monotonically  increasing  epochs,  and 

ifoi  =  o)  =  (sw,  pt,  pkt),  i.e.  N  —4  . . .  — - >  Nj  ■J+1> 

. . .  Nk,  then  no  o(  £  {o(,  ■  ■  ■  ,  precedes  oi,  and 
the  oi  transition  is  an  OUT  terminating  at  a  host. 


Unconstrained  single-packet  traces  are  not  required  to  begin  at  a 
host.  We  write  T(N)  for  the  set  of  unconstrained  single-packet 
traces  generated  by  N,  and  note  that  T (N)  C  T (N). 

Definition  9  (Network  Kripke  Structure).  Let  N  be  a  static  net¬ 
work.  We  define  a  Kripke  structure  IC(N)  =  (Q,  Q  o,  <5,  A)  as  fol¬ 
lows.  The  set  of  states  Q  comprises  tuples  of  the  form  (sw,pt,  Tk). 
The  set  Qo  contains  states  (sw,  pt,  Tk)  where  sw  and  pt  are  adja¬ 
cent  to  an  ingress  link — i.e.,  there  exists  a  link  Lj  and  host  h  such 
that  Lj  .loc  =  h  and  Lj  .loc'  =  (sw,pt).  Transition  relation  5  con¬ 
tains  all  pairs  of  states  (sw,pt,  Tk)  and  (sw'  ,pt' ,  Tjf)  where  there 
exists  a  switch  S  and  a  link  L  such  that  S.sw  =  sw  and  either: 

•  there  exists  a  link  Lj  and  packets  pkt  £  Tk  and  pkt'  £  Tj. 
such  that  L.loc'  =  (sw,pt)  and  Lj.loc  =  (sw,ptj)  and 
Lj.loc1  =  (sw',pt')  and  (pktfptj)  £  \S.tbl\(pkt,pt). 

•  there  exists  a  link  Lj,  a  host  h,  and  packets  pkt  £  Tk  and 
pkt '  £  Tj.  such  that  L.loc'  =  (sw,pt)  and  Lj.loc  =  (sw,pt') 
and  Lj.loc'  =  h  and  (pkt' ,pt')  £  \S.tbl\(pkt,pt). 

•  (sw,  pt,  Tfc)  =  (sw' ,  pt' ,  Tj)  and  there  exists  a  packet  pkt  £ 
Tfc  such  that  L.loc'  —  (sw,pt)  and  \S.tbl\(pkt,  pt)  =  {}. 

•  (sw,pt,Tk)  =  (sw' ,pt' ,Tj)  and  there  exists  a  link  Lj  and 
host  h  such  that  Lj.loc  =  (sw,  pt)  and  Lj.loc'  =  h. 

Finally,  the  labeling  function  A  maps  each  state  (sw,pt,Tk)  to 
Tfc,  which  captures  the  set  of  all  possible  header  values  of  packets 
located  at  switch  sw  and  port  pt. 


The  four  cases  of  the  S  relation  correspond  to  forwarding  packets  to 
an  internal  link,  forwarding  packets  out  an  egress,  dropping  packets 
on  a  switch,  or  reaching  an  egress  (inducing  a  self-loop). 

We  can  relate  the  observations  generated  by  a  network  N  and 
the  traces  of  the  Kripke  structure  generated  from  it. 

Definition  10  (Trace  Relation).  Let  N  be  a  static  network  and 
K  a  Kripke  structure.  Let  <  be  a  relation  on  observations  of  N 
and  states  of  K  defined  by  (sw,pt,pkt)  <  (sw,pt,Tk)  if  and 
only  if  pkt  £  Tk.  Lift  <  to  a  relation  on  (finite)  sequences  of 
observations  and  ( infinite )  traces  by  repeating  the  final  observation 
and  requiring  <  to  hold  pointwise:  o\  ■  ■  ■  Ofc  <  t  if  and  only  if 
°i  <  ti  for  i  from  1  to  k  and  ok  <  tj  for  all  j  >  k. 

Lemma  4  (Traces  of  a  Stable  Network).  Let  N  be  a  stable  network. 
Then  for  each  trace  t  £  T  (N),  there  exists  a  trace  t'  £  T  (N)  such 
that  t  is  a  suffix  oft'. 

Lemma  5  (Trace-Equivalence).  Let  Ni ,  Nn  be  static  networks 
where  N i  —>•••—>  Nn  and  no  transition  is  an  update  command. 
For  a  single-packet  trace  t,  we  have  t  £  T(Nf)  t£'T(N„). 

Lemma  6  (Induced  Sequence  of  Networks).  Let  N\  be  a  static 
network,  and  let  N[  be  the  network  obtained  by  emptying  all 
packets  from  Ni.  Let  cmds  be  a  sequence  of  commands,  and  let 
Ci  •  •  •  cn- 1  be  the  subsequence  of  update  commands.  Construct  the 
sequence  N[  —t  •  •  •  — >  Nj  of  empty  networks  by  executing  the 
update  commands  in  order.  Now,  given  any  sequence  Ni 
Nn  induced  by  cmds,  we  have  Ni  ~  N [  for  all  i. 

In  other  words,  any  induced  sequence  of  static  networks  is  point- 
wise  trace-equivalent  to  the  unique  sequence  of  network  configura¬ 
tions  generated  by  running  the  update  commands  in  order. 


B.  Synthesis  Algorithm  Correctness  Proofs 

Lemma  1  (Network  Kripke  Structure  Soundness).  Let  N  be  a 
static  network  and  K  —  IC(N)  a  network  Kripke  structure.  For 
every  single-packet  trace  t  in  T (N)  there  exists  a  trace  t'  of  K 
from  a  start  state  such  that  t  <  t' ,  and  vice  versa. 


Proof.  We  proceed  by  induction  over  k,  the  length  of  the  (finite 
prefix  of  the)  trace.  The  base  case  k  —  1  is  easy  to  see,  since 
the  lone  observation  in  t  must  be  on  an  ingress  link,  meaning  the 
corresponding  state  in  K  will  be  an  initial  state  with  a  self-loop 
(case  3  of  Definition  9),  and  these  are  equivalent  via  Definition  10. 

For  the  inductive  step  (k  >  1),  we  wish  to  show  both  directions 
of  subtrace  relation  <  to  conclude  equivalence.  First,  let  t  = 
o i,  •  •  •  ,  Ofc+i  be  a  single-packet  trace  of  length  k  +  1  in  T(N), 
and  we  must  show  that  3t'  £  IC(N)  such  that  t  <  t' .  Let  tk 
be  the  prefix  of  t  having  length  k.  By  our  induction  hypothesis, 
there  exists  t'h  =  Si,--  -  ,  Sfc_i,  sk,  sk,  ■  ■  •  £  N(N)  such  that 
tk  t'k  ■  We  have  the  successor  relation  ok  U  ok+ 1,  so  Definition 
7  and  9  tells  us  that  we  have  a  transition  st  — >  s'  for  some 
s'  £  K.  We  see  that  this  s'  is  exactly  what  we  need  to  construct 
t'  =  si,  •  •  •  ,  Sfc,  s',  s',  ■  ■  ■  which  satisfies  the  relation  t  <  t' . 

Now,  let  t'  =  si,  •  •  •  ,  Sfc,  Sfc+i,  Sfc+i,  •  •  •  be  a  trace  in  IC(N) 
for  which  the  finite  prefix  has  length  k  +  1.  We  must  show  that 
3 1  £  T(N)  such  that  t  <  t' .  Let  t'k  =  si,  •  •  •  ,  Sfc_i,  sk,  Sk,-  •  • , 
and  by  our  induction  hypothesis,  and  there  exists  tk  =  0\,  ■  ■  ■  ,0h 
such  that  tk  <  t'k .  Consider  transition  sk  — >  Sfc+i.  If  Sk  =  Sfc+i, 
then  t'  =  t'k ,  so  we  can  let  t  =  tk,  and  conclude  that  t  <  t' . 
Otherwise,  if  sk  Sfc+i,  then  we  have  one  of  the  first  two  cases 
in  Definition  9,  which  correspond  to  the  cases  in  Definition  7, 
allowing  us  to  construct  an  ok+ i  such  that  ok  U  Ofc+i.  We  let 
t  =  oi,  •  •  •  ,0k,  Ok+ 1,  and  conclude  that  t  <t' .  □ 
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We  want  to  develop  a  lemma  showing  that  the  correctness  of  careful 
command  sequences  can  be  reduced  to  the  correctness  of  each 
induced  TV, ,  so  we  start  with  the  following  auxiliary  lemma: 

Lemma  7  (Traces  of  a  Careful  Network).  Let  N  be  a  stable 
network  with  C.  cmds  careful,  and  consider  a  sequence  of  static 
networks  induced  by  C.cmds.  For  every  trace  t  £  7~(TV)  there 
exists  a  stable  static  network  Ni  in  the  sequence  s.t.  t  £  T{Ni). 

Proof.  I.  First,  we  show  that  at  most  one  update  transition  can  be 

°1  °'k 

involved  in  the  trace.  In  other  words,  if  TV  — >  . . .  — >  Nk  where 
t  =  o\  •  •  •  on  is  a  subsequence  of  o\  ■  ■  ■  o'k,  and  if  /  :  N  — >  N  is  a 
bijection  between  Oi  indices  and  o'  indices,  then  at  most  one  of  the 
transitions  o^(1),  •  •  •  ,  o}^  is  an  UPDATE  transition. 

Assume  to  the  contrary  that  there  are  more  than  one  such 
transitions,  and  consider  two  of  them,  o\,o'j  where  i,j  £ 
{/( 1),  •  •  •  ,  /(«•)},  assuming  without  loss  of  generality  that  i  <  j. 
Now,  since  the  sequence  C.cmds  is  careful,  we  must  have  both  an 
INCR  and  Flush  transition  between  o'  and  o' .  This  means  that  the 
second  update  o'  cannot  happen  while  the  trace’s  packet  is  still  in 
the  network,  i.e.  j  >  f(n),  and  we  have  reached  a  contradiction. 

II.  Now,  if  there  are  zero  update  transitions,  we  are  done,  since 
the  trace  is  contained  in  the  first  static  TV.  If  there  is  one  update 
transition  Nk+i  =  Nk[sw  ■£-  tbl],  and  this  update  occurs  before 
the  packet  reaches  sw  in  the  trace,  then  the  trace  is  fully  contained 
in  Nk- 1-1.  Otherwise,  the  trace  is  fully  contained  in  Nk.  □ 

Lemma  2  (Careful  Correctness).  Let  N  be  a  stable  network  with 
C.cmds  careful  and  let  p  be  an  LTL  formula.  If  cmds  is  careful 
and  Ni  |=  f>  for  each  static  network  in  any  sequence  induced  by 
cmds,  then  cmds  is  correct  with  respect  to  p. 

Proof.  Consider  a  trace  t  £  T(N).  From  Lemma  7,  we  have 
t  £  T(Ni)  for  some  Ni  in  the  induced  sequence.  Thus  t  \=  ip, 
since  our  hypothesis  tells  us  that  Ni  |=  ip.  Since  this  is  true  for 
an  arbitrary  trace,  we  have  shown  that  T(TV)  |=  p,  i.e.  N  \=  ip, 
meaning  that  cmds  is  correct  with  respect  to  p.  □ 

Theorem  1  (Soundness).  Given  initial  network  Ni,  final  configu¬ 
ration  Nf,  and  LTL  formula  p,  if  OrderUpdate  returns  a  com¬ 
mand  sequence  cmds,  then  Ni  c—>  TV'  s.t.  TV'  ~  TV/,  and  cmds 
is  correct  with  respect  to  p  and  Ni. 

Proof.  It  is  easy  to  show  that  if  OrderUpdate  returns  cmds,  then 
Ni  '—r  TV'  where  TV'  ~  TV/.  Each  update  in  the  returned  sequence 
changes  a  switch  configuration  of  one  switch  s  to  the  configuration 
Nf(s),  and  the  algorithm  terminates  when  all  (and  only)  switches 
s  such  that  TV;(s)  ^  TV/(s)  have  been  updated. 

Observe  that  if  ORDERUPDATE  returns  cmds,  the  sequence  can 
be  made  careful  by  choosing  an  adequate  time  delay  between  each 
update  command,  and  for  all  j  £  {0,  •  •  •  ,  n}.  TV/  |=  p.  This  is 
ensured  by  the  call  to  a  model  checker  (Line  7).  We  use  Lemma  2 
to  conclude  that  cmds  is  correct  with  respect  to  p  and  Ni.  □ 

To  show  that  ORDERUPDATE  is  complete  with  respect  to  simple 
and  careful  command  sequences,  we  observe  that  ORDERUPDATE 
searches  through  all  simple  and  careful  sequences. 

Theorem  2  (Completeness).  Given  initial  network  Ni,  final  con¬ 
figuration  Nf,  and  specification  p,  if  there  exists  a  simple,  careful 
sequence  cmds  with  TV;  TTfp  TV'  s.t.  TV'  ~  TV/,  then  ORDERUP¬ 
DATE  returns  one  such  sequence. 


C.  Incremental  Checking  Correctness  Proofs 

Lemma  3.  First,  H olds  Sink  (q,  AT)  4A  3 1  £  traces{q)  :  t  \=  M 
for  sink  states  q.  Second,  if  labGrK  is  a  correct  labeling  with 
respect  to  p  and  succK(q),  then  Holds K(q,  AT,  labGr k)  -4=>- 
3/  £  traces K{q)  :  t  |=  M. 

Proof.  First,  for  sink  states,  observe  that  there  is  a  unique  trace  t 
in  traces(q),  as  q  is  a  sink  state.  We  first  prove  that  t  \=  p  iff 
Holdso(q,  p).  We  prove  this  by  induction  on  the  structure  of  the 
LTL  formula.  Then  we  observe  that  there  is  a  unique  maximally- 
consistent  set  M  such  that  t  \=  AT.  This  is  the  set  {ip  \  t  |=  ipAip  £ 
ecl{p)}.  We  then  use  the  definition  of  HoldsSink{q ,  M)  for  sink 
states  to  conclude  the  proof. 

Now  consider  non-sink  states:  we  first  prove  soundness,  i.e.,  if 
Holdsx{q ,  AT,  labGrK),  then  there  exists  t  £  traces(q)  such  that 
t  |=  TV/.  We  have  HoldsK{q,  AT,  labGrK )  iff  (\(q)  =  (APC\M)) 
and  there  exists  q'  £  succk{M),  and  AT'  £  labGr  K{q')  such  that 
follows(M,  M').  By  assumption  of  the  theorem,  we  have  that  if 
AT'  £  labGr  K(q'),  then  there  exists  a  trace  t'  in  traces{q')  such 
that  f'  |=  A/'.  Consider  a  trace  t  such  that  to  =  q  and  t1  =  t! . 
For  each  ip  £  AT,  we  can  prove  that  t  |=  ip  as  follows.  The  base 
case  of  the  proof  by  induction  is  implied  by  the  fact  that  q  \= 
( AP  (T  A/).  The  inductive  cases  are  proven  using  the  definitions  of 
maximally-consistent  set  and  the  function  follows.  We  now  prove 
completeness,  i.e..  that  if  there  exists  a  trace  t  in  traces  K{q)  such 
that  t.  |=  AT,  then  HoldsK(q,  AT,  labGrK )  is  true.  Let  t.  be  the 
trace  qqiq2  ....  It  is  easy  to  see  that  if  M  is  a  maximally-consistent 
set,  and  t  |=  AT,  then  AT  =  {ip  \  ip  £  ecl{p)  A  t  |=  ip}.  Let  us 
consider  the  set  of  formulas  S  =  {ip  \  ip  £  ecl(p)  A  f1  |=  ip}. 
Observe  that  S'  is  a  maximally-consistent  set.  By  assumption  of  the 
theorem,  we  have  that  S  is  in  labGr  K{qi).  It  is  easy  to  verify  that 
follows{AT,  S).  □ 

Theorem  3.  Let  V  C  Q  be  a  set  of  vertices  and  labGrK  a 
correct  labeling  with  respect  to  p  and  Q  \  ancestors k{V).  Then 
relbli({p,  labGrK ,  V)  is  a  correct  labeling  w.r.t.  p  and  Q. 

Proof.  We  first  note  that  only  ancestors  of  nodes  in  V  are  re¬ 
labeled — all  the  other  nodes  are  correctly  labeled  by  assumption 
on  labGr.  We  say  that  a  node  q  is  at  level  k  w.r.t.  a  set  of  vertices 
T  iff  the  longest  simple  path  from  q  to  a  node  in  T  is  k.  Let  Hk 
be  the  set  of  nodes  at  level  k  from  V .  We  prove  by  induction  on  k 
that  at  fc-th  iteration,  we  have  a  correct  labeling  of  K  w.r.t.  p  and 
(S  \  ancestors k(V))  U  Hk,  where  S  is  the  set  of  states  of  K.  We 
can  prove  the  inductive  claim  using  Lemma  3.  □ 

Corollary  1.  First,  modelCheckK{p)  =  true  K  \= 

p.  Second,  for  (K,K',U)  and  labGrK  as  above,  we  have 
incrModelCheck(K,  p,U,  labGrK)  =  true  4=^  K  |=  p. 

Proof.  Using  Theorem  3,  and  the  fact  that  the  set  ancestors K{Sf) 
is  the  set  S  of  all  states  K,  we  obtain  that  labGrK  = 
relbNip,  labGr°K,  S /)  is  a  correct  labeling  of  K  with  respect 
to  p  and  S.  In  particular,  for  all  initial  states  qo,  we  have  that 
for  all  AT  C  ecl(p),  m  £  labGr K^qo)  iff  there  exists  a  trace 
t  £  tracesK^qo )  such  that  t  |=  AT.  We  now  use  the  definition  of 
checklnitStates  to  show  that  if  checklnitStates  returns  true,  then 
there  is  no  initial  state  qo  such  that  there  exists  AT  £  labGr k (qo) 
such  that  ->p  £  M.  Thus  for  all  initial  states  qo,  for  all  traces  t  in 
traces  {to),  we  have  that  t  |=  p. 

The  proof  for  incremental  model  checking  is  similar.  □ 
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